FastForward for Efficient Pipeline Parallelism ; CU-CS-1028-07

High-rate core-to-core communication is critical for efficient pipeline-parallel software architectures. This paper presents the FastForward system, a software-only lowoverhead high-rate queue implementation for pipeline parallelism on multicore architectures. FastForward uses an architecturally-tuned domain-specific adaptation of concurrent lock-free queues to provide low-latency and lowoverhead core-to-core communication. Enqueue and dequeue times on a 2 GHz Opteron 270 based system are as low as 36 ns, up to 4x faster than the next best solution. FastForward’s effectiveness is demonstrated for real applications by applying it to network processing, resulting in record-breaking throughput for commodity hardware. A proof of correctness shows that FastForward works on strong to very weakly ordered consistency models.